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ABSTRACT 

The purpose of developmental evaluation is to guard 
against over-simplification. Three questions which the evaluator may 
ask to help accomplish this task are: 1) What are the conditions 
under which the project will have to exist? 2) What are the 
activities or transactions that will have to take place if the 
project is to be successful? 3) What outcomes, both intended and 
unintended, are the logical consequences of the project? (JK) 
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To begin with an allusion, and a literary one at that, is 
probably wise when you're faced with a topic which is itself 
something of an allusion. Some would say illusion, I suppose, 
and be in some measure correct. The topic is clearly allusive 
in that I'm sure it conjures up in some minds images of an 
evaluative history characterized by evaluators who promised more 
than they could deliver and by designs that proved to be inadequate 
for the questions they were suppose to answer or the decisions 
they were intended to inform. Some people who have experienced 
the mandate and press for evaluation in the several titles of 
the Elementary avid Secondary School Education Act, for example, 
in state-wide gifted programs, and broad social programs across 
the country may some day call this the age of the great evaluation 

hoax. 

The picture of present practice in evaluation is not pretty. 

The evaluation specialist still struggles for a methodology, still 
debates the raison d'&tre of his own roles and goal. At the same 
time students, teachers, and administrators, across the land continue 



*The title alludes to Robert E. Stake's-"The Countenance of 
Educational Evaluation," Teachers College Record , 68:7:43-60. Apr. 1967. 

An earlier version of this paper was presented at the 1972 
Association for Educational Communications and Technology annual 
convention in Minneapolis. 
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to request evaluation expertise. Whether this press for service 
should be construed as an affirmation of some rational model of 
man— some implicit belief that better information is related to 
better decisions— or as a response of a conmunity become more 
sensitive about its limited resources and the significance of 
worth (both perhaps or neither) is a question deserving of atten- 
tion. To put the question more simply is to ask the first of a 
series of key questions. That question is: 

Why do we or should we evaluate? 

The answer to that one is bound up in a second one: 

What is an evaluation supposed to do? 

When we have a momentary answer to both of these questions, we 
will have constructed a rationale for evaluation, and that is the 
first and most important problem that has to be solved. 

Who do we or should we evaluate? 

What is an evaluation supposed to do? 

In what follows, I will offer one answer to the question, "What 
is an evaluation supposed to do?" In so doing, I will try to present 
one important idea, and three additional questions that may help us 
move from theory into practice, a bloody hard thing to accomplish. 

Let me begin now the pursuit of an idea. Let me see if I can 
find a language and a metaphor that will mean it to you. I shall 
begin with simplification. 

More than 100 years ago, the Swiss historian, Jacob Burckhardt 
saw that ours would be the age of the great simplification and that 
this simplification would come to be the damning peril of the times. 
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These have become meaningful words for me as I have tried to practice 

evaluation within a development process. 

My friend, Bob iamond, who labors with me at the Center for 
Instructional Development at Syracuse University, has often reminded 
me that part of my role as an evaluator is to keep the Center honest. 

I am still not quite sure what that means. I'm somewhat more sure 
that it's a hard thing to do. More seriously, if I accept Burckhardt's 
observation that over-simplification is a damning peril, I then want 
to say that part of dishonesty lies in the denial of complexity. Now, 
I'm getting closer to the idea. 

To deny complexity is to be dishonest. What the evaluator has 
to learn how to do is to guard against over-simplification, and to 
do this means to be able to describe complexity. To borrow a phrase 
from the researcher, within the development process the evaluator 
must work to avoid the type one error or the too quick rejection of 
the null hypothesis that says: no difference. 

Let me try to illustrate what I mean when I say that developmental 
evaluation works to guard against over-simplification. The process 
of curriculum or course development can be described as an arduous, 
logical attempt to define some conditions, specify a range of treatments 
or activities, and posit a set of outcomes or things that may happen 
as a result of the activities. Drawing from previous research, experi- 
ence, and at times, good strong hunches, the curriculum developer will 
attempt to state--as specifically as he dare— the logical relationships 
that describe the module, course, or program he intends. Those who 
bear the scars of this process can testify to its difficulty, profit, 
and comedy. 
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Given this logical array, or seeking to find it out, what 
the evaluator must do is develop a series of data sets that 
will allow judgments to be made as to whether or not the inten- 
tions of development have been fulfilled in practice. It is in 
this way that the evaluator will guard against over-simplification. 

He will guard against the notion that wishing makes it so. 

Examples of this type of developmental evaluation have 
characterized current efforts at the Center for Instructional 
Development. Some few may help me clarify the idea of evaluation 
as a guard against simplification. 

In one project, it was intended that students would enter the 
course already possessing certain pre-requisite knowledges and 
skills that would be necessary for additional instruction to be 
effective. Confidence on the part of the content expert was high. 

He was quite certain that his students would possess these knowledges 
and skills. He had stated a logical condition. As part of the 
developmental evaluation, measures were collected to determine if 
that was the state of affairs. Contrary to expectation, the data 
suggested that as many as 80% of the entering students did not 
possess these skills and knowledges. Consequently, a package of 
remedial materials was developed. We had guarded against the early 
acceptance of an assumption by submitting it to empirical examination. 

In another project, a small lecture-discussion session was 
supposed to be characterized by high levels of verbal interaction 
and an crphasis on the cognitive processes of analysis, synthesis, 
and evaluation, an intended activity. Data collected said that 
approximately 75% of the class time had been devoted to straight 
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lecture presentation and that students judged the emphasis in 
that series of lectures to be on factual recall and the didactic 
presentation of definitions and rules. Again, we had been able 
to guard against the naive acceptance of activity or treatment 
by identifying, to some extent, where the logical intention had 
not been fulfilled in practice. 

One last example. In another course the intention was chat 
after treatment students would exhibit positive attitudes toward 
both the treatment and the course as a whole. Evidence gathered 
here through questionnaires, rating scales, and interviews suggested 
that not only did students not like the treatment but more dis- 
arming, their attitudes toward the course and the content area 
itself were considerably more negative at the finish than they 
were at the outset. Very upsetting news indeed. 

I think these are three examples of how evaluation was used 
during development to guard against over-simplification and what 
you might like to think of as the self-fulfilling prophecy of 
curriculum development. 

Lastly, there are three questions that I hope will provide 
some great big hooks (some might rather say hang-ups) that will 
help put your evaluation concerns in perspective. In the realm 
of rationale, I have already asked: 

Why do we or should we evaluate? 

and 

What is an evaluation supposed to do? 
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In the realm of practice, I would like to add these three key 
questions: 

What are the conditions under which the project 
will have to exist? 

What are the activities or transactions that will 
have to take place if the project is to he 

successful? 

What outcomes 3 both intended and unintended 3 are 
the logical consequences of the project? 

In one sense, the roles of the developer and the evaluator 
can come as close together as a tense change from do to did. They 
certainly are as close as the difference between asking, "What, 
shall we do?" and "What shall I say we did?" These are surely 
some of the key questions of development as much, if not more so, 
than they are the questions of the evaluator. It may be that one 
of the unintended outcomes of this consideration is that we have 
inched a little closer to a realization that the logical relation- 
ships that unite development and evaluation are so strong as to 
deny any willful separation of the two, very much as my two eyes 
make one in sight. 

Possibly, some developers have lived too long in the kingdom 
of the blind where they have been one-eyed, half-sighted, but 
nevertheless, kings. Possibly, what I have been talking about is 
what two eyes can do toward bringing us closer to true perspective, 
toward helping us deny simplification and visualize complexity. 



